Survey Participation, Nonresponse Bias, Measurement Error Bias, and Total Bias

A common hypothesis about practices to reduce survey nonresponse is that those persons brought into the respondent pool through persuasive efforts may provide data filled with measurement error. Two questions flow from this hypothesis. First, does the mean square error of a statistic increase when sample persons who are less likely to be contacted or cooperate are incorporated into the respondent pool? Second, do nonresponse bias estimates made on the respondents, using survey reports instead of records, provide accurate information about nonresponse bias? Using a unique data set, the Wisconsin Divorce Study, with divorce records as the frame and questions about the frame information included in the questionnaire, this article takes a first look into these two issues. We find that the relationship between nonresponse bias, measurement error bias, and response propensity is statisticspecific and specific to the type of nonresponse. Total bias tends to be lower on estimates calculated using all respondents, compared with those with only the highest contact and cooperation propensities, and nonresponse bias analyses based on respondents yield conclusions similar to those based on records. Finally, we find that error properties of statistics may differ from error properties of the individual variables used to calculate the statistics. 737 Published in Public Opinion Quarterly 70:5 (Special Issue 2006), pp. 737–758; doi 10.1093/ poq/nfl038 Copyright © 2006 Kristen Olson. Published by Oxford University Press on behalf of the American Association for Public Opinion Research. Used by permission. http:// poq.oxfordjournals.org/cgi/reprint/70/5/737 Kristen Olson is a graduate student in the program of survey methodology at the University of Michigan. This article is part of the author’s doctoral dissertation research and is based on work supported by the National Science Foundation under Grant no. SES-0620228. The author is especially grateful to her committee—Bob Groves, T. E. Raghunathan, Rod Little, Yu Xie, and Norman Bradburn—for discussion and insight into the problem. Frauke Kreuter and Sonja Ziniel provided comments on an earlier draft that improved the article immensely. The author is indebted to Vaughn Call and Colter Mitchell for providing access to the Wisconsin Divorce Study. The Wisconsin Divorce Study was funded by a grant (HD-31035 and HD32180–03) from the National Institute of Child Health and Human Development, the National Institutes of Health. The study was designed and carried out at the Center for Demography and Ecology at the University of Wisconsin–Madison and Brigham Young University under the direction of Vaughn Call and Larry Bumpass. 738 Kr i s t e n Ol s O n i n Pu b l i c OP i n i O n Qu a r t e r l y 70 (2006)